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DETAILED ACTION 

Amendments and Remarks 

1 . Applicant's amendments and remarks dated 9 April 2008, have been entered and made of 
record. 

Response to Amendments and Remarks 

2. Applicant's amendments to claims 1, 9, and 17 are sufficient to overcome the previous § 
102 rejections. Accordingly, those rejections have been withdrawn. However, in view of newly- 
discovered prior art, new grounds of rejection appear below. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1-23 are rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. Patent 
6,806,898 by Toyama et al. ("Toyama") in view of U.S. Patent 6,707,933 by Mariani et al. 
("Mariani"). 

Regarding claim 1 , Toyama discloses a method (figure 8) for processing an image 
containing at least a portion of a head of a human in a video phone system, comprising: 

estimating an orientation of said head in said image using a pattern recognition technique 

(810); 
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computing a three dimensional model of a face surface of said human using a computer 
vision technique (822, 824); and 

adjusting an orientation of said three dimensional face surface model to provide a frontal 
view (826). 

Toyama does not appear to disclose that the image is kept unmodified when the 
orientation of the head is estimated to be frontal, as claimed. It appears that Toyama performs the 
computing and adjusting steps regardless of the orientation of the head. 

Mariani discloses a facial direction estimation system. In particular, for a video 
conferencing environment, Mariani teaches that "[ujsing the face direction estimation ... it is 
possible to enhance the quality of the transmitted images by generating a frontal view when the 
face is off-frontal ... or by unchanging the current satisfying frontal face which is displayed" 
(column 3/34-40). Thus, Mariani, like Toyama, recognizes the need to adjust the orientation of 
the face in order to achieve eye-to-eye contact for video conferencing communications. Mariani 
further recognizes that when the original image of the face is a "satisfying frontal face" there is 
no need to change the orientation of the head, and the image can be left unchanged. Accordingly, 
it would have been obvious to one of ordinary skill in the art at the time of the invention to 
modify Toyama by Mariani to achieve the claimed invention since Mariani teaches that when the 
orientation of the head is estimated to be front-facing, there is no need to modify the image of the 
face since it is already in the desired orientation. 
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Regarding claim 2, Toyama discloses said computing step further comprises the step of 
using a symmetric face assumption to obtain a complete three dimensional face surface model 
for a profile view (824). 

Regarding claim 3, Toyama discloses said computing step further comprises the step of 
employing a structure from motion technique to obtain said three dimensional face surface model 
(column 13/1-10). 

Regarding claim 4, Toyama discloses said estimating step applies a classification 
technique (i.e. determining the orientation of the head necessarily involves a technique that 
classifies, i.e., ascertains, the pose of the head using any known method — column 6/53-67). 

Regarding claim 5, Toyama discloses said computing step generates a morphable three 
dimensional model (column 10/60 — 1 1/43: the 3D model of the face can be morphed, i.e., 
rotated, moved, changed, etc., into a desired shape/pose). 

Regarding claim 6, Toyama discloses the step of mapping said three dimensional face 
surface model having an adjusted orientation to a two dimensional space (828). 

Regarding claim 7, Toyama discloses the step of transmitting said adjusted image to a 
remote user (see e.g. figure 2). 

Regarding claim 8, Toyama discloses the step of presenting said adjusted image to a local 
user (see e.g. figures 1 and 2). 

Regarding claim 9, Toyama discloses an image processor (102, figure 1) for use in a 
video phone system, comprising: 
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a memory (104) for storing an image containing at least a portion of a head of a human; 

and 

a head pose corrector that 

(i) estimates an orientation of said head in said image using a pattern recognition 
technique (810, figure 8); 

(iii) computes a three dimensional model of a face surface of said human using a 
computer vision technique (822, 824, figure 8); and 

(iv) adjusts an orientation of said three dimensional face surface model to provide 
a frontal view (826, figure 8). 

Toyama does not appear to disclose that the image is kept unmodified when the 
orientation of the head is estimated to be frontal, as claimed. It appears that Toyama performs the 
computing and adjusting steps regardless of the orientation of the head. 

Mariani discloses a facial direction estimation system. In particular, for a video 
conferencing environment, Mariani teaches that "[u]sing the face direction estimation ... it is 
possible to enhance the quality of the transmitted images by generating a frontal view when the 
face is off-frontal ... or by unchanging the current satisfying frontal face which is displayed" 
(column 3/34-40). Thus, Mariani, like Toyama, recognizes the need to adjust the orientation of 
the face in order to achieve eye-to-eye contact for video conferencing communications. Mariani 
further recognizes that when the original image of the face is a "satisfying frontal face" there is 
no need to change the orientation of the head, and the image can be left unchanged. Accordingly, 
it would have been obvious to one of ordinary skill in the art at the time of the invention to 
modify Toyama by Mariani to achieve the claimed invention since Mariani teaches that when the 
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orientation of the head is estimated to be front-facing, there is no need to modify the image of the 
face since it is already in the desired orientation. 

Regarding claim 10, Toyama discloses said head pose corrector is further configured to 
use a symmetric face assumption to obtain a complete three dimensional face surface model for a 
profile view (824, figure 8). 

Regarding claim 11, Toyama discloses said head pose corrector is further configured to 
employ a structure from motion technique to obtain said three dimensional face surface model 
(column 13/1-10). 

Regarding claim 12, Toyama discloses said head pose corrector is further configured to 
apply a classification technique to obtain said head orientation (i.e. determining the orientation of 
the head necessarily involves a technique that classifies, i.e., ascertains, the pose of the head 
using any known method — column 6/53-67). 

Regarding claim 13, Toyama discloses said three dimensional face surface model is a 
morphable three dimensional model (column 10/60 — 1 1/43: the 3D model of the face can be 
morphed, i.e., rotated, moved, changed, etc., into a desired shape/pose). 

Regarding claim 14, Toyama discloses said head pose corrector is further configured to 
map said three dimensional face surface model having an adjusted orientation to a two 
dimensional modified image (828, figure 8). 

Regarding claim 15, Toyama discloses said two dimensional modified image is 
transmitted to a remote user (see e.g. figure 2). 
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Regarding claim 16, Toyama discloses said two dimensional modified image is presented 
to a local user (see e.g. figures 1 and 2). 

Regarding claim 17, Toyama discloses a video phone system (figures 1-2), comprising: 
a memory (104, figure 1) for storing an image containing at least a portion of a head of a 

human; and 

a head pose corrector (102, figure 1) that 

(i) estimates an orientation of said head in said image using a pattern recognition 

technique (810, figure 8); 

(iii) computes a three dimensional model of a face surface of said human using a 
computer vision technique (822, 824, figure 8); and 

(iv) adjusts an orientation of said three dimensional face surface model to provide 
a frontal view (826, figure 8). 

Toyama does not appear to disclose that the image is kept unmodified when the 
orientation of the head is estimated to be frontal, as claimed. It appears that Toyama performs the 
computing and adjusting steps regardless of the orientation of the head. 

Mariani discloses a facial direction estimation system. In particular, for a video 
conferencing environment, Mariani teaches that "[u]sing the face direction estimation ... it is 
possible to enhance the quality of the transmitted images by generating a frontal view when the 
face is off-frontal ... or by unchanging the current satisfying frontal face which is displayed" 
(column 3/34-40). Thus, Mariani, like Toyama, recognizes the need to adjust the orientation of 
the face in order to achieve eye-to-eye contact for video conferencing communications. Mariani 
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further recognizes that when the original image of the face is a "satisfying frontal face" there is 
no need to change the orientation of the head, and the image can be left unchanged. Accordingly, 
it would have been obvious to one of ordinary skill in the art at the time of the invention to 
modify Toyama by Mariani to achieve the claimed invention since Mariani teaches that when the 
orientation of the head is estimated to be front-facing, there is no need to modify the image of the 
face since it is already in the desired orientation. 

Regarding claim 18, Toyama discloses said head pose corrector is further configured to 
use a symmetric face assumption to obtain a complete three dimensional face surface model for a 
profile view (824, figure 8). 

Regarding claim 19, Toyama discloses said head pose corrector is further configured to 
employ a structure from motion technique to obtain said three dimensional face surface model 
(column 13/1-10). 

Regarding claim 20, Toyama discloses said head pose corrector is further configured to 
apply a classification technique to obtain said head orientation (i.e. determining the orientation of 
the head necessarily involves a technique that classifies, i.e., ascertains, the pose of the head 
using any known method — column 6/53-67). 

Regarding claim 21, Toyama discloses said head pose corrector is further configured to 
map said three dimensional face surface model having an adjusted orientation to a two 
dimensional modified image (828, figure 8). 

Regarding claim 22, Toyama discloses said two dimensional modified image is 
transmitted to a remote user (see e.g. figure 2). 
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Regarding claim 23, Toyama discloses said two dimensional modified image is presented 
to a local user (see e.g. figures 1 and 2). 

Conclusion 

5. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Colin M. LaRose whose telephone number is (571) 272-7423. If 
attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Brian 
Werner, can be reached on (571) 272-7401. The fax phone number for the organization where 
this application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
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applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. Any inquiry 
of a general nature or relating to the status of this application or proceeding can also be directed 
to the TC 2600 Customer Service Office whose telephone number is (571) 272-2600. 



/Colin M. LaRose/ 
Colin M. LaRose 
Primary Examiner 
Group Art Unit 2624 
26 June 2008 



